Relative Density Nets: A New Way to Combine Backpropagation with HMM's
نویسندگان
چکیده
Geoffrey E. Hinton Gatsby Unit, UCL London, UK WCIN 3AR [email protected] Logistic units in the first hidden layer of a feedforward neural network compute the relative probability of a data point under two Gaussians. This leads us to consider substituting other density models. We present an architecture for performing discriminative learning of Hidden Markov Models using a network of many small HMM's. Experiments on speech data show it to be superior to the standard method of discriminatively training HMM's.
منابع مشابه
Wavelet-based non-parametric HMM's: theory and applications
In this paper, we propose a new algorithm for non-parametric estimation of hidden Markov models (HMM's). The algorithm is based on a \wavelet-shrinkage" density estimator for the state-conditional probability density functions of the HMM's. It operates in an iterative fashion, similar to the EM re-estimation formulae used for maximum likelihood estimation of parametric HMM's. We apply the resul...
متن کاملProbabilistic Backward and Forward Reasoning in Stochastic Relational Worlds
Inference in graphical models has emerged as a promising technique for planning. A recent approach to decision-theoretic planning in relational domains uses forward inference in dynamic Bayesian networks compiled from learned probabilistic relational rules. Inspired by work in non-relational domains with small state spaces, we derive a backpropagation method for such nets in relational domains ...
متن کاملDecoupled Learning for Conditional Adversarial Networks
Incorporating encoding-decoding nets with adversarial nets has been widely adopted in image generation tasks. We observe that the state-of-the-art achievements were obtained by carefully balancing the reconstruction loss and adversarial loss, and such balance shifts with different network structures, datasets, and training strategies. Empirical studies have demonstrated that an inappropriate we...
متن کاملSolving Fuzzy Equations Using Neural Nets with a New Learning Algorithm
Artificial neural networks have the advantages such as learning, adaptation, fault-tolerance, parallelism and generalization. This paper mainly intends to offer a novel method for finding a solution of a fuzzy equation that supposedly has a real solution. For this scope, we applied an architecture of fuzzy neural networks such that the corresponding connection weights are real numbers. The ...
متن کاملA predictor-corrector method for the training of deep neural networks
The training of deep neural nets is expensive. We present a predictorcorrectormethod for the training of deep neural nets. It alternates a predictor pass with a corrector pass using stochastic gradient descent with backpropagation such that there is no loss in validation accuracy. No special modifications to SGD with backpropagation is required by this methodology. Our experiments showed a time...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2001